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Abstract 

In this work we compute the thermodynamic properties of the 3-satisfiability problem in the infinite connectivity limit. 
In this limit the computation can be strongly simplified and the thermodynamic properties can be obtained with a high 
accuracy. We find evidence for a continuous replica symmetry breaking in the region of high number of clauses, a > a c . 
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1 Introduction 

The statistical mechanics of the random if -satisfiability (K-SAT) problem has been the object of many studies in the 
last years Q| g{ . The K-SAT was the first problem to be shown to be Non-deterministic Polynomial (NP) complete Q . 
This model is important because it provides a simple prototype for all the NP complete problems in complexity theory 
of computer science as well as in statistical mechanics of disordered and glassy systems, in computational biology and in 
other fields. 

It is usually believed that the solutions of NP complete problems, or the certainty that they have no solutions, can 
only be found, in the worst case, by algorithms with a running time of computation that grows faster than polynomially 
(namely exponentially) with the number of variables N of the system. 

Generally speaking, in the statistical mechanics approach, for each given instance of the problem, one introduces a 
Hamiltonian Ti(C), constructed in such a way that the configuration C* , which minimize Ti(C), is the solution of the 
problem if TL(C*) = 0. On the contrary, if Tt(C) > for any C, the problem does not have a solution. In this framework 
one consider for each problem the partition function 

Z(/3) = ^exp(-/W(C)) , (1) 

a 

where f3 = T _1 , T being the temperature of the system. In the same way one introduces the usual thermodynamic 
quantities, e.g. the internal energy 

m—t^SL m 

It can be argued that quantities like the internal energy density (i.e. E = B/N) do not depend on N in the infinite N 
limit, so that a computation of their average over the different instances of the problem is sufficient for obtain interesting 
information in this limit. 

When one studies the behavior of the K-SAT model at finite temperature, one finds a rich structure of phase transitions 
M. In certain region of the parameter space, replica symmetry is broken (in other words there are many equilibrium states 
in the large volume limit M). Explicit computation shows that it may be possible to obtain a basic understanding of 
the connection between the SAT/UNSAT phase transition in random combinatorial structures (l[Q and the transition 
between a Replica Symmetric (RS) structure and a structure where the replica symmetry is broken in the frame of spin- 
glasses jB|. Recent results 0] suggest that the SAT/UNSAT transition seems to take place into the phase of broken replica 
symmetry. Yet the question stays open of how exactly the typical-case complexity theory of computer science and the 
Replica Symmetry Breaking (RSB) transition are related. 

One of the main aim of the recent research on this model has been to understand better the structure of solutions, 
especially at the borderline between the region of the phase diagram where the problem has solution and the region where 
no solution is possible. Indeed this is the zone where the most unlikely (hardest) solutions are. 

We will concentrate our attention on the case K — 3 (3-SAT), that is known to be the first and simplest NP complete 
instance of K-SAT. The 2-SAT model is, in fact, already solvable in a time increasing polynomially (actually even linearly 
M) with the number of variables. 

The basic boolean variables of the problem, s(i), are defined on the sites i with i = 1, N. For technical reasons we 
prefer to use variables a(i) which take the values ±1. Our problem is then related to the original one through the variables 
transformation s(i) = (1 + er(i))/2. 



We consider an ensemble of randomly generated 3-SAT formulae. The Hamiltonian corresponding to a given formula 

is 

^ i _ ^•"■ < 3) g( j l) i _ ^^^ a{l2) 1 _ r „. 
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For each instance of the problem we generate aN clauses, where each clause is determined by randomly selecting three of 
the N sites and assigning to them a random ±1 variable. The terns of randomly chosen sites {11,12,23} are given by the 
variables Ti ±1 i 2l i 3 that take the value 1 with probability p = aN~ 2 and the value with probability 1 — p. For finite N 
there are approximately aN variables r which are different from zero and they become exactly equal to aN in the limit 
N — > 00. Given a tern {11,12,23}, a set of three variables e are drawn, taking the value +1 or —1 with probability 1/2. 
The function (^) depends only on those variables e ('i' l 2> ! 3) such that Ti lt i 2 ,i 3 = 1 and all the terms are non-negative: Ti 
just counts the number of clauses that are not satisfied. Obviously Ti = if and only if all the clauses are satisfied. 

As we have already explained we are going to consider Ti as the Hamiltonian of a disordered system, in order to 
apply to the K-SAT model the statistical mechanics techniques and to compute all the mathematical expressions in this 
framework. We will introduce the Active temperature T = 1//3 and at the end we will send f3 — + 00 to compute the ground 
state properties. 

In the large N limit we can write the equivalent Hamiltonian, in which the number of terms in the interaction is fixed 
and equal to aN, as: 

v 1- 4'M4°) i-4'M4°) 1-4V4 ) m 
2 2 2 { ' 
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where the sites if* (t — 1, 2, 3) are randomly chosen for each one of the aN triplets. 

For reasons that are discussed in (l[j2[, one is interested to study the statistical properties of the system in the 
thermodynamic limit. It is interesting to consider the zero temperature energy density Eo(a) (i.e. the average over the 
distribution of clauses of the number of clauses that are not satisfied by the formula corresponding to the Hamiltonian in 
equation (^)) as a function of the ratio a between the number of clauses and the number of variables. We are eventually 
interested in the zero temperature entropy density So (a). The number of solutions satisfying the formula is asymptotically 
given by exp(NSo(a)). It has been conjectured JlJ that 

Eo(a) = 0, So (a) > 0, for a < a c , (5) 
So (a) = 0, E (a) > 0, for a > a c , 

where the value of a c is estimated to be around 4.2 ||. 

Below a c we have solutions (with probability going to 1 for N going to infinity), while above a c the problem does 
not have solutions (i.e. it is UNSAT). At a <C a c the problem is quite underconstrained and it is relatively easy to find 
an assignment of variables {<Ji} satisfying the clauses. For a ^> a c , though in general still hard, to prove unsatisfiability 
is easier than in the hardest cases near a c . Around the density of clauses a c it is indeed very difficult either to find a 
satisfying assignment or to show unsatisfiability, i.e. it is most difficult to discriminate whether the problem admit any 
solution or no solution at all. These are the cases where an exponential time may be needed. 

Far from this critical value, anyway, things simplify and more insight over the structure of the phase space can be 
gained. 

Indeed the exact evaluation of the free energy in the a-/3 plane is a rather complex computation. The aim of the present 
work is then to show that the computation strongly simplifies in the most overconstrained limit of a — * 00. Let us first 
introduce the reduced inverse temperature {1 through the relation 

0=-£=. (6) 

and let us define the rescaled energy density 

e(p,a) S -^(£(Aa)-f) , (7) 

A similar definition can be written for the other thermodynamic functions. In particular for the free energy we have the 
rescaled quantity /(/1, a) = (F((3, a) — a/8)/y/a. We shall also introduce the reduced temperature r = /i -1 = Ta 1 ' 2 . We 
will show below that the function e(fi, a) has a limit when a goes to infinity. We can thus define 

eQtt) = lim e(/i, a). (8) 

a — >oo 

In the following we will also compute the function e(/x) with high accuracy. In the conclusions we will implicitly assume 
that the limit a — > 00 of full connectivity and the zero temperature (/1 — > 00) limit can be freely exchanged. 
The interest for this computation is threefold: 

• The limit where a goes to infinity plays the same role of the infinite connectivity model for spin glasses (finite 
connectivity /dilute models correspond to finite a) and most of our analytic understanding comes from the study 
of the infinite range models (Sherrington-Kirkpatrick like models Q), where the analytic computations are much 
simpler. 

• Replica symmetry is broken in a region of the a-/3 plane, for a > a c ~ 4.2 j| (or maybe for a > a s ~ 3.9 as recently 
derived in j(| by means of a variational approach). It is reasonable to assume that in this whole region the way in 
which replica symmetry is broken is the same as in the limit a — > 00. 



• If we neglect the dependence of eo(a) = lim^^oo e(fi, a) on a for a > a c (i.e. if we perform an asymptotic expansion 
in 1/s/a and we consider only the leading order), we get the following estimate for a c : 

a c w (8e ) 2 . (9) 

Where the estimate a c depends from the order of the asymptotic expansion. In this work we will limit ourselves to 
the leading order in the expansion. The precise value of eo will be given in the next section where we derive the 
thermodynamic observables using the replica tool. 



2 The Replica Formalism 

In the replica formalism one computes 

Z {n) ee Z[{A}}» , (10) 

where {A} denotes the random couplings, the bar is the average over the distribution of the random couplings and the 
partition function is defined as: 

Z[{A}} = ]T exp(-/3H[Wi}, {A}]) . (11) 

In our case A represents the ensemble of random clauses, namely the ensemble of terns {ii, ii, 13} with associated e's. The 
free energy density at finite n is defined as 

In te) 

F w = - lim \ Ar ' . (12) 

N^oa /3nN v ' 

where we firstly perform the thermodynamic limit keeping n fixed and only afterwards we send n — > by an analytic 
continuation procedure. 

We are eventually interested in computing the limit n — > of F'"', which is the value of the free energy density of the 
generic system in the infinite volume limit: 



F = lim fW = - lim lnZ }j A}] . (13) 



Starting from (H) and carrying out the average over the distribution of the r's we have 



*i>»2j*a V V o=l,nj=l,3 / / 

where here the (...) is now the average only over the e's distribution. In the limit of large N we can write the previous 
expression in terms of the effective Hamiltonian 7i e [[ and the temperature like parameter a: 

= ^exp(-aH e// ) , (15) 

where 

Ue ff = Tf2 ^2 heff H* 1 ),^^),^)!/?) (16) 



TV 2 



and 



ft e// EEl-exp -/3^ J] —2 J' (17) 

For a given tern of sites {11,12,23} the average is performed over the 2 3 possible values of the three variables e. 

Let us now consider the limit a — *■ 00 at fixed /j,. The computation is long but straightforward. In this limit the inverse 
temperature /3, defined as in ([]), becomes a quantity of order a- 1 ' 2 so that we can freely expand the exponential in the 
previous expression (|l6|): 
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,3 



zi<Z2<*3 V a= 1,71 j = 1,3 \a — l,nj — 1,3 



where 



I _ ,(il.*2,i3)_oC,' 
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a = l,n j — 1,3 



En 1 - - !En^44Enc^i <»> 

\a — 1 , n j = 1 , 3 / a.fe j-1,3 a<.b j— 1,3 



Using standard manipulations, for large N and a, we easily get 



Z (n) =exp 



= exp 
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In the infinite N limit we can use the saddle point method and we find that the free energy density is given by 

/5 



F(a) = -^A[{Q ap },{A sp }] 



and the expression of the internal energy comes out to be: 



^)^T( 1+ iE(i+^) 3 

\ a<b 



The elements {Q sp } and {A sp } satisfy the following consistency equations: 



ab 64 



sp\2 
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(22) 



(23) 



(24) 



(25) 



Our task is now to find the solution of these equations. 

We notice that equation (B4|) implies that the exact definition of eo in (pi) is 



eo 



lim a 

fl^OO 8 



1 + ^E( 1+( ?- 



a<b 



(26) 



3 The Replica Symmetric Solution 



The simplest possibility (which is correct at high temperature) consists in assuming that the off-diagonal elements of the 
matrix Q and A are constant and they are equal to qo and Ao respectively. A simple computation shows that 



A(q ,X ) = 



16 ~ 128 (1+</o) 2 + 



(27) 



+ / dp(zo) log ^2 cosh za \f\oj 
and the final form of the rescaled free energy, as defined in (Q), is given by 



/(/•«> a) =- 



16 



(l + qo) 2 (2-g ) 



+— /dp(zo)log^2coshzo\/Ao^ 



where the parameters 50 and Ao satisfy the equations: 

Ao = ^^ 2 (l + qo) 2 , qo = J dp(zo) (taxlh.(z^/\o~)j , 
and we have used the following compact measure for the Gaussian measure: 



dp[z) 



2 /2 
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■dz 



(28) 



(29) 



(30) 



The solution of the equation (£9|) can be found by iterations. The parameter qo is an analytic function of r (fig. 3, full 
curve). No transition is present. The zero temperature value of the rescaled energy (Q) is given by eo = v/gjf ~ .345494. 

The corresponding value of a where the energy density E([i, a) goes to zero, i. e. the ratio of the number of variables 
and the number of clauses up to which all clauses are satisfied, turns out to be, following ([]), a c = 24/ n = 7.6394373 in 
this asymptotic approximation. 

The value of the entropy corresponding to this solution is shown in figure 1. It becomes negative at low temperature 
signaling an inconsistency of the approach based on replica symmetry. In order to obtain reasonable results in the low 
temperature region we must break the replica symmetry as we will show in the next section. 
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Figure 1: Entropy as function of the reduced temperature r in the replica symmetric case and in the broken symmetry case 
with one and two steps breaking. 
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Figure 2: Detail of the cases with broken replica symmetry, shown in figure [I], at low temperature. 

4 The Replica Symmetry Breaking 
4.1 One step Replica Symmetry Breaking 

If replica symmetry is broken, very often reasonable results are obtained in the framework of the one step replica symmetry 
breaking, where it is assumed that the elements of the matrix Q take only two values (for the physical interpretation of 
one step replica symmetry breaking see reference. JBj). 

In the one step case one divides the indices a in n/m groups, each group having m components. We set Q a ,b equal to 
q 1 if a and b belong to the same group, otherwise we set Q a ,t equal to go. Similar relations are used for the matrix A. The 
free energy is now a function of three independent parameters: m, qo and q\. The limit n — > is obtained in this case by 
doing an analytic continuation also in m, that in such a process will not be integer anymore. In the replicae limit m 
acquires non-integer values between and 1. 

After some simple computation we get 



A(q ,qi;Xo,Xi,m) = 



(31) 



+ JE - T28 H 1 + 9o) 3 + (1 - -)(1 + 9if] + 
~ + i[mA ( )go + (1 - m)Aigi] + 

d P(^o)log^y dp(zi) ^2cosh^o\/A^+zi\/Ai - AoJ^j 



(32) 



!-?((! + 9«) 2 (1 - 2go) - (1 + gi) 2 (l - 2,1)) + 
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(l+ gi ) 2 (2- gi ) 



mjlf dp ( z °) log y ( 2 cosh (z Q \f\o + z 1 \/Ai — A () 



where the following equations are satisfied: 



A i = ^/i 2 (l + g i ) 2 , i = 0,l 



9o 



9i = 



J" dp(zi) (cosh («o\/Ao + zi\Ai — -^o))" 1 

x j dp(zi)tanh^2 '\/Ao+ 2 i'\/^i^^()) (cosh^z \/Ao+2i \/ Ai— Ao)^ 

i 

dp(zo 



/ dp(zi) (cosh (zo\/Ao + 2i\/A7^Ao)) m 
y dp(zi) ^anh^2oV^o+2i"\/ Ai— Aojj (cosh^oV'Ao+zi y/Xi— Aojjj 



(33) 
(34) 

(35) 



and the parameter m is chosen in such a way to minimize the resulting free energy. 

One finds that for /i > /j, c = 4.55 the previous equations have a non trivial solution (e.g. m / 0,?i / qo)- The 
corresponding values of m, qo and q\ are shown in figures 3 and 4. ft is evident that the value of m is different from 
the one at critical temperature and that the difference q\ — qo vanishes at fj, c . This behavior is similar to the one that is 
realized in the Sherrington-Kirkpatrick (SK) model in non-zero magnetic field. 




Figure 3: Values of qo as function of the temperature r in the replica symmetric case (full curve), of go & n d qi in the broken 
symmetry case with one step breaking (dashed curves) and those of qo , qi and qi in the two step replica symmetry breaking 
case (dotted curves). 



In figures 1 and 2 we plot the entropy as function of the temperature. Also in this case the entropy becomes negative 
at sufficiently small temperature, but this happens in a rather smaller region above r = 0. 
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Figure 4: Parameters m's as function of r for one step (full curve) and two steps (dashed curves) replica symmetry breaking. 



In the SK model for spin glasses, where this disaster happens at one step level, (we recall that the entropy cannot 
be negative), the correct value of the entropy is proportional to r 2 . If a similar behaviour is present in this model the 
free energy should be given by A + Bt a at small, but not too small r. We show in figures 5 and 6 the free energy as 
function of r 3 . We see that in a wide range of r 3 a linear behaviour is present supporting a quadratic dependence of the 
entropy on the the temperature. The extrapolated value of the zero temperature rescaled free energy obtained using this 
method (/£,t = A — —0.333412) is slightly larger than the actual value at zero temperature (/irsb(T = 0) = —0.333740), 
however this first value should be more reliable, because it is known that the errors in the free energy, if one makes the 
approximation of considering only a finite number of RSB steps, are negligible at higher temperature, but they become 
larger at low temperature. f~£) t gives an estimate Q equal to a c ~ 7.114468 
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Figure 5: Rescaled free energy versus the cube of the reduced temperature t 3 in the replica symmetric case and in the broken 
symmetry cases with one and two steps breaking. 



4.2 Two steps Replica Symmetry Breaking 

In we perform another step in breaking the replica symmetry and we let the elements of Q and A take three different 
values (qo, qi and q^ and Ao, Ai and A2 respectively) the free energy will be a function of the five independent variables: 
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Figure 6: Detail of figure [| showing the one and two step symmetry breaking cases at low r. 



qo, 31, 92, mi, 7712. In this case we have: 

A(q , 91,92; A (go), Ai(gi), A2O72)) 
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min 

where the self consistency equations are 



1 - - (mi ((1 + go) 2 (l - 2q ) - (1 + gi) 2 (l - 2gi)) + 
+m 2 ((l + gi ) 2 (l-2 gi )-(l + g 2) 2 (l-2 g2 )))- (1 + ^ (2 ' g2) 

' dp(z ) x 

dp(z 2 ) ^cosh^o^/Ao+^iy 7 Xi—Xo+Z2y/ A 2 -A 



+ 



Ai = ^m 2 (i + a0 2 , 



90 = /dp(z ) 



X / dp(zi) 
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dp(z 2 ) ^cosh^o\Ao+ 2 i\Ai - X +Z 2 y/X 2 - Xijj m2 



m 2 x 



x / dp(z 2 ) tanh ^Z(r\Ao+ 2 i\A 1 — Ao +Z2\fx 2 - X ij x 

x ^cosh^zo-\/Ao+2iV / Ai-Ao+z2Y / A 2 -AiJ^ ^ 
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dp(zi) 



dp(z2) (cosh(zo^/Xo+z 1 y / X 1 -\ +Z2\/\2-^ 

dp(z 2 ) ^cosh ^zo\Ao +zi-y/Si - Ao+Z2V / A2 - AiJ) m2 x 
X tanh(z 
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92= / dp(z ) 
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X / dp(zi) 
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(41) 



From figures 3 and 4 we see that the transition between the replica symmetric structure and the broken one at 
t c — l/fi c = 0.21978 is confirmed. In the two step computation the entropy (fig. 2) still becomes negative but at a lower 
temperature than in the one step case and the zero temperature value is less negative than before. Even the r 2 behaviour 
for small r, or the equivalent A + Br 3 law for the free energy, is satisfied up to a smaller temperature. The extrapolated 

f 2^ 

value of the zero temperature free energy is now y e J t = A = —0.333401, where the value given by the actual 2RSB free 
energy is / 2BSB = —0.333450. Their difference is of an order of magnitude smaller than in the one step replica symmetry 
breaking case (we recall that in that case f£\ = -0.333412 and f 1RSB (T = 0) = -0.333740). 

The pathologies here exhibited clearly show that the one step and the two steps solutions are still not exact, and it is 
natural to suppose that they will disappear when we break the replica symmetry in a continuous way [^]. 

Using equation (^) once again, we find a c ~ 7.1139985. This value is not too far from the estimated result (i.e. 4.2 H) 
if we consider how crude is our approximation. 



5 Conclusions 

We have seen that the transition from the replica symmetric case to the replica broken case is a smooth transition, which 
is quite different from the quasi first order transition of the p-spin model [B] . This difference is likely due to the fact that 
the self overlap go is different from zero in the high temperature phase. 

The one step approximation is not exact at low temperature and it is likely not to be exact in the whole replica 
symmetry broken phase. The two step replica symmetry breaking computation gives clear hints that the replica symmetry 
must be broken in a continuous way. In any case both approximations apparently give excellent approximations for the 
free energy at low temperature (the error on the value of the zero temperature free energy is O(10 -4 ) at one step and 
O(10 ) at two steps RSB level respectively). 

The model has a behaviour that is very similar to the one of the Sherrington-Kirkpatrick model in presence of a 
magnetic field. It is natural to conjecture that these properties hold in a quite large interval of values of a, for a > a c , 
even far from the full connectivity limit a — > oo. It would be very interesting to check these predictions using numerical 
simulations and/or analytic tools. 

We wish to thank Enzo Marinari and Riccardo Zecchina for many useful discussions on the subject of this paper. 
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